This invention relates generally to touch display systems, and more particularly, to generating correction parameters used by algorithms to locate run-time touches on touch display systems.
Touch displays and touch display systems are provided for an increasing number of applications. Point of sale (POS), for processing transactions within a department store, and point of information (POI), such as an electronic directory are common. For example, applications include airport passenger and baggage check-in, and kiosks located within a store that provide information about products and services. Furthermore, touch displays are increasingly used in handheld computing applications such as for rental car check-in, package delivery services, and stockroom inventory management.
A touch display system has a display for visually presenting data to a user. A touchscreen is installed in front of the display. The user interacts with the system by touching the touchscreen at locations over software programmed buttons or icons in the displayed image or by moving a cursor via a moving touch. With the broad use of graphical user interfaces (GUI) such as Microsoft Windows® that provide many small buttons, menu selections, and the like, it is increasingly important for touchscreen measured locations to correspond with the desired cursor locations in the display image. A number of calibration methods have been developed to properly map touchscreen coordinates to display coordinates. As used herein, “touchscreen coordinate system” refers to coordinate locations detected as a result of a touch on the touchscreen, and “display coordinate system” refers to coordinate locations of points on the display. In practice, touchscreen software communicates coordinates to an operating system that has its own coordinate system that in turn is mapped to the coordinate locations of the points on the display. Therefore, the term “display coordinate system” is generalized herein to include operating system coordinate systems.
Linear and non-linear distortions may be present within the displays. Linear distortion may be one or more of horizontal and vertical offsets as well as scale change or magnification. An example of linear distortion is when equally spaced straight lines remain equally spaced straight lines. With non-linear distortion, equally spaced straight lines are distorted such that lines are no longer straight nor equally spaced, and may also have offset and scale distortions. For example, images on cathode ray tube (CRT) displays are subject to non-linear distortions. Today, however, flat panel displays with minimal non-linear distortion have largely replaced CRT displays and may include liquid crystal displays (LCDs), plasma displays, electroluminescent displays, and organic light-emitting diode (OLED) displays, etc. The discussion herein assumes that the display coordinate system is free of non-linear distortions.
Ideally, touchscreen hardware would be perfectly linear, generating x and y coordinates that differ from display coordinates by, at most, linear distortions. Even small touchscreen non-linear distortions may be a problem for GUI operation via a touch display system. Such non-linear distortions of raw touchscreen coordinates may be due to touchscreen manufacturing variations. Alternatively, the touchscreen may be non-linear by design as a consequence of other design objectives such as narrow borders or reduced manufacturing cost. Due to the non-linearities, there is demand for calibration methods to map non-linear touchscreen coordinates to linear display coordinates to assure that touches activate intended GUI icons.
There are many types of touchscreens that may be used in touch display systems. Touchscreen types include surface acoustic wave touchscreens, infrared touchscreens, capacitive touchscreens and 4-wire, 5-wire and 9-wire resistive touchscreens. 5-wire touchscreens are a common choice. The basic principles of 5-wire touchscreen operation and forming of the touchscreen substrate are described in the patent literature, such as in U.S. Pat. No. 6,593,916 (Elo Touchsystems, Inc., 2003).
In the construction of a typical 5-wire resistive touchscreen, a polymer film such as polyethylene terephthalate (PET) forms a coversheet that is placed over a glass substrate with a narrow air gap in between. In the absence of a touch, spacers prevent contact between the coversheet and glass substrate. The two surfaces facing the air gap are coated with transparent conductive coatings such as indium tin oxide (ITO). Associated electronics and interconnections alternately apply X and Y voltage gradients to the coating on the glass substrate. The coating on the coversheet is connected to voltage sensing circuitry. A touch causes the coversheet to deflect and contact the substrate thus making a localized electrical connection between the conductive coatings. As a result of this electrical contact, the local voltage on the glass substrate's coating is communicated to the coversheet coating which is connected to voltage sensing circuitry. One voltage is measured when an X voltage gradient is applied to the glass substrate's coating and another voltage is measured for a Y voltage gradient. These two voltages are the raw touch (x,y) coordinate data. The voltage gradients are not perfect and may lead or contribute to non-linear and/or linear distortions.
To form an exemplary 5-wire touchscreen substrate, a complex electrode geometry composed of discrete electrodes is fabricated around the perimeter of the glass substrate. Such 5-wire touchscreen designs may be referred to as discrete-electrode 5-wire resistive touchscreens or simply discrete-electrode touchscreens. Discrete electrode touchscreens have a discrete number of electrical connections to the resistive coating that covers the touch area. Very close to the perimeter of the touch area, the discreteness of these connections results in distortions of the lines of constant voltage. In contrast, well away from the perimeter, the lines of constant voltage can be straight and equally spaced and represent the ideal linear touchscreen. In some cases, in areas of a discrete-electrode touchscreen away from the perimeter, a small, yet undesirable, amount of non-linearity may be experienced.
An alternative 5-wire touchscreen may be referred to as a picture-frame touchscreen and is discussed in U.S. Pat. No. 6,650,319 (Elo Touchsystems, Inc., 2003). A resistive electrode in continuous contact with the resistive coating surrounds the touch area much like a picture frame surrounds a picture. The resistive picture-frame electrode results in curved lines of constant voltage or global bowing distortions. Picture-frame touchscreens are quite non-linear by design and depend on software correction of non-linearity. The picture-frame touchscreen design does provide a number of benefits including narrow borders as well as the elimination of the perimeter distortions seen in discrete-electrode touchscreen design.
Known methods exist for mapping touchscreen coordinates to display coordinates, such as 3-point calibration that may be used for linear mappings. More sophisticated calibration and mapping methods also exist that account for non-linear distortions within raw touchscreen coordinates.
U.S. Pat. No. 5,751,276 (Microsoft Corporation, 1998) proposes an alternate linear calibration method in which redundant data is collected and linear correction coefficients are determined from a least squares fit. With the addition of two more calibration constants, linear shear distortions and arbitrary rotations of the touchscreen coordinate system with respect to the display coordinate system can be corrected. However, it is still a linear correction scheme in which equally spaced straight lines remain equally spaced straight lines.
For many applications, 2-point or 3-point calibration is sufficient. This is often the case for applications that only involve user selection of large icons or touch buttons. However, other applications are more demanding, such as graphical user interfaces (GUI) that require precise cursor control to properly activate numerous small icons, buttons, scroll bars, etc. For such demanding applications, imperfections in the touchscreen hardware may lead to unacceptable non-linear distortions of the touch system coordinates that cannot be corrected with the linear 3-point calibration. Furthermore, if the touchscreen is non-linear by design, such as in a picture-frame touchscreen, the linear 3-point calibration method is insufficient.
Calibration methods involving more than three points that correct for non-linear distortions are also known. For example Japanese patent application publication 2005-134992 (application JP 2003-367561) of Gunze proposes a 9-point calibration scheme to determine coefficients of non-linear corrections for picture-frame touchscreens. 25-point calibration procedures are also used.
Turning to a 25-point calibration, let (X1, Y1), (X2, Y2), . . . (X25, Y25) be the locations in the display coordinate system of 25 calibration targets imaged on the display device. The 25 calibration points may, for example, be arranged in a 5 by 5 grid. As used herein, (X,Y) are display system coordinates and (x,y) are touchscreen system coordinates. As the calibration targets are displayed one by one on the touch display, the user touches the calibration targets, resulting in measured touchscreen coordinates (x1, y1), (x2, y2), . . . (X25, Y25). In mathematical notation, for calibration target index k=1, 2, . . . 25, measured touchscreen coordinates (xk, yk) correspond to display coordinates (Xk, Yk). Such calibration data is used to generate correction coefficients or parameters for run-time algorithms. The calibration targets may represent predetermined locations or calibration coordinates within the display coordinate system. Therefore, the calibration targets need not be displayed on a corresponding and/or attached display device. For example, during manufacturing an automated method may be used to touch the predetermined locations on the touchscreen prior to mounting the touchscreen and the display device together. Examples of automated methods include a stencil, a bed-of-nails, an activation device that is robotically controlled, and the like.
There are several options for extracting correction coefficients or parameters from the 25-point calibration data. For example the calibration data could be used to construct look-up table data to be used in combination with linear interpolation. Polynomial expansions are also used to generate mappings.
Polynomial expansions may be accomplished with respect to the center of the display. Let (xC,yC) be the touchscreen coordinates generated by a touch at the center of the touch display. Third-order polynomial expansions for display coordinates (X,Y) in terms of touchscreen coordinates (x,y) are given by Equation (Eq.) 1 and 2 below. Each equation contains ten monomial terms each with its own coefficient for a total of twenty calibration constants A0,0, A1,0, A0,1, A2,0, A1,1, A0,2, A3,0, A2,1, A1,2, A0,3, B0,0, B1,0, B0,1, B2,0, B1,1, B0,2, B3,0, B2,1, B1,2 and B0,3. For example, polynomial coefficients may be computed from the calibration data (Xk, Yk) and (xk, yk) for k=1 to 25.
                    X        =                              A                          0              ,              0                                +                                    A                              1                ,                0                                      ·                          (                              x                -                                  x                  C                                            )                                +                                    A                              0                ,                1                                      ·                          (                              y                -                                  y                  C                                            )                                +                                    A                              2                ,                0                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            2                                +                                    A                              1                ,                1                                      ·                          (                              x                -                                  x                  C                                            )                        ·                          (                              y                -                                  y                  C                                            )                                +                                    A                              0                ,                2                                      ·                                          (                                  y                  -                                      y                    C                                                  )                            2                                +                                    A                              3                ,                0                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            3                                +                                    A                              2                ,                1                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            2                        ·                          (                              y                -                                  y                  C                                            )                                +                                    A                              1                ,                2                                      ·                          (                              x                -                                  x                  C                                            )                        ·                                          (                                  y                  -                                      y                    C                                                  )                            2                                +                                                    A                                  0                  ,                                      3                    ·                                                              ⁡                              (                                  y                  -                                      y                    C                                                  )                                      3                                              Eq        .                                  ⁢        1                                Y        =                              B                          0              ,              0                                +                                    B                              0                ,                1                                      ·                          (                              x                -                                  x                  C                                            )                                +                                    B                              0                ,                1                                      ·                          (                              y                -                                  y                  C                                            )                                +                                    B                              2                ,                0                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            2                                +                                    B                              1                ,                1                                      ·                          (                              x                -                                  x                  C                                            )                        ·                          (                              y                -                                  y                  C                                            )                                +                                    B                              0                ,                2                                      ·                                          (                                  y                  -                                      y                    C                                                  )                            2                                +                                    B                              3                ,                0                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            3                                +                                    B                              2                ,                1                                      ·                                          (                                  x                  -                                      x                    C                                                  )                            2                        ·                          (                              y                -                                  y                  C                                            )                                +                                    B                              1                ,                2                                      ·                          (                              x                -                                  x                                      C                    ⁢                                                                                                                            )                        ·                                          (                                  y                  -                                      y                    C                                                  )                            2                                +                                    B                              0                ,                3                                      ·                                          (                                  y                  -                                      y                    C                                                  )                            3                                                          Eq        .                                  ⁢        2            
For convenience, Eq. 3 and 4 assume that the touchscreen coordinates have been offset to move the origin to the center, that is, xC=0 and yC=0. The general offset case may be returned to by the substitutions x→(x−xC) and y→(y−yC).X=A0,0+A1,0·x+A0,1·y+A2,0·x2+A1,1·xy+A0,2·y2+A3,0·x3+A2,1·x2y+A1,2·xy2+A0,3·y3  Eq. 3Y=B0,0+B1,0·x+B0,1·y+B2,0·x2+B1,1·xy+B0,2·y2+B3,0·x3+B2,1·x2y+B1,2·xy2+B0,3y3  Eq. 4
The notation may be further simplified by using Σ summation notation where N=3 for the 3rd order polynomial expansion of Eq. 3 and 4.
                    X        =                              ∑                          m              ≥                              0                ⁢                n                            ≥              0                                                      m                +                n                            ≤              N                                ⁢                                    A                              m                ,                n                                      ·                          x              m                        ·                          y              n                                                          Eq        .                                  ⁢        5                                          Y          =                                                    ∑                                                      m                    +                    n                                    ≤                  N                                                            m                ≥                                  0                  ⁢                  n                                ≥                0                                      ⁢                                          B                                  m                  ,                  n                                            ·                              x                m                                                    ⁣                  y          n                                    Eq        .                                  ⁢        6            Eq. 5 and 6 generalize to polynomial expansions of different orders depending on the value of N. For example, for values of N of 3, 5 and 7, the number of terms in each of the above equations is 10, 21 and 36 respectively. For a 7th order expansion, the total number of calibration constants Am,n and Bn,m is 2*36=72, exceeding the number (2*25=50) of measured values xk and yk (k=1 to 25) from the 25-point calibration process. In contrast, the number of constants for 5th and lower order polynomial expansions is less than the number of 25-point calibration measurements.
The polynomial coefficients Am,n and Bn,m may be determined by doing a least squares fit. For any proposed set of polynomial coefficients Am,n and Bn,m, one may compute the difference between Xk, the known X coordinate of calibration target k used to display the calibration target, and the value of X computed above for the corresponding measured touchscreen coordinates (xk, yk). The discrepancy between actual and computed horizontal coordinates may be notated as ΔXk and given by Eq. 7. ΔYk may be similarly defined.
                              Δ          ⁢                                          ⁢                      X            k                          =                              X            k                    -                                                    ∑                                  m                  ≥                                      0                    ⁢                    n                                    ≥                  0                                                                              m                  +                  n                                ≤                N                                      ⁢                                          A                                  m                  ,                  n                                            ·                              x                k                m                            ·                              y                k                n                                                                        Eq        .                                  ⁢        7            The polynomial coefficients Am,n and Bn,m may be determined by minimizing the sum of squares (SS) of Eq. 8, namely minimizing the SS of the discrepancies. (As the first sum depends only on the parameters Am,n and the second sum only on the parameters Bn,m, each sum can be minimized separately with the same result.) Algorithms and code for such least squares fits are well known and widely available.
                    SS        =                                            ∑                              k                =                1                            25                        ⁢                                          (                                  Δ                  ⁢                                                                          ⁢                                      X                    k                                                  )                            2                                +                                    ∑                              k                =                1                            25                        ⁢                                          (                                  Δ                  ⁢                                                                          ⁢                                      Y                    k                                                  )                            2                                                          Eq        .                                  ⁢        8            This method for determining polynomial coefficients that are used to calibrate the mapping between touchscreen coordinates (x,y) and display coordinates (X,Y) may be referred to as a conventional general polynomial fit to Nth order. Herein the conventional general polynomial fit may also be referred to as a conventional general fit. A conventional general polynomial fit to 1st order is a linear fit. A 3rd order conventional general polynomial fit is often sufficient to correct for slight bowing or pin-cushion non-linearities due to manufacturing variations of a nominally linear touchscreen design. Higher order polynomial fits are desirable to handle stronger non-linearities such as those of picture-frame touchscreens. The 25-point calibration process based on a conventional general polynomial fit is more powerful than the common 3-point linear calibration method but still has limitations.
In the conventional general polynomial fit, display coordinates (X,Y) are represented as polynomial expansions of the touchscreen coordinates (x,y). Such expansions are directly usable by run-time code that receives raw touchscreen coordinates (x,y) as input and generates display coordinates (X,Y) as output. Parameters from the polynomial fit, such as the monomial coefficients Am,n and Bn,m, may be directly stored by the calibration process for later use by run-time code during operation of the touch display system.
The conventional general polynomial fit severely limits the maximum order of the polynomial expansion. For example, to keep the number of calibration constants smaller than the number of measurements from a 25-point calibration grid, the order of the polynomial expansion must be no larger than five, that is, N≦5. Increasing the calibration grid to, for example, 9×9 results in a total of 2*81=162 measured values and may allow expansions up to order N=11. However, with a 9×9 calibration grid each of the 81 calibration targets must be touched.
The above assumes that the calibration targets are touched with perfect accuracy, which is almost never the case. Measured calibration data (xk, yk) are subject to electronic noise. Additionally, imprecise activation of touch calibration target locations may also result in noise. With noisy data, having a greater number of fit parameters can result in a worse fit. In general, fits with high degrees of redundancy (many more data points than fit parameters) filter measurement noise while fits with low degrees of redundancy (number of data points not much larger than or equal to the number of fit parameters) amplify measurement noise particularly when extrapolating beyond the data range.
For a least squares fit of a polynomial expansion to 25-point calibration data, there are 25 x measurements and 25 y measurements for a total of 50 measured values. For the 3rd order polynomial expansion discussed above, there are 10 fit parameters Am,n and 10 fit parameters Bn,m for a total of 20 fit parameters. For 25-point calibration the redundancy is high (50 is much larger than 20) and the fitting process statistically averages out much of the calibration measurement noise. In contrast, for a 5th order polynomial expansion, there are 21 Am,n parameters and 21 Bn,m parameters for a total of 42. For 25-point calibration the redundancy is low (50 is not much larger than 42) and the fit makes poor predictions particularly when extrapolating to locations outside the 5×5 grid. Hence in practice, even 5th order is problematic for a conventional general polynomial fit to 25-point calibration data.
For picture-frame touchscreens, the 5×5 calibration grid can be expanded to be essentially the same size as the touch sensitive touch area. This eliminates the need to extrapolate to locations outside the calibration grid where conventional general polynomial fits tend to be most problematic. In contrast, for discrete-electrode touchscreens, the perimeter of the touch area is subject to complex local distortions as discussed above. The local distortions further add to the noise level of the touchscreen coordinates. To avoid this additional source of noise, all 5×5 calibration targets are positioned away from the perimeter and thus create a need for reliable extrapolation outside the 5×5 calibration grid.
Therefore, a need exists for an improved calibration method for touch display systems.